simulate rsvps for realtime quickstart #8180
Conversation
Codecov Report
@@ Coverage Diff @@
## master #8180 +/- ##
============================================
- Coverage 71.41% 64.68% -6.74%
- Complexity 4302 4303 +1
============================================
Files 1623 1578 -45
Lines 84312 82431 -1881
Branches 12639 12435 -204
============================================
- Hits 60213 53319 -6894
- Misses 19974 25301 +5327
+ Partials 4125 3811 -314
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
richardstartin
changed the title
richardstartin
force-pushed
the
retire-quickstart-rsvps
branch
from
5aeccef
to
1b1cd4c
Compare
| session.addMessageHandler(String.class, getMessageHandler()); | ||
| } | ||
| }, ClientEndpointConfig.Builder.create().build(), new URI("wss://stream.meetup.com/2/rsvps")); | ||
| _source.start(); |
There was a problem hiding this comment.
i felt like this original construct was to demonstrate how to create a listening client against some public API and redirect data into pinot.
If we are not doing such it this can simply be a periodic task that runs _producer.produce(createMessage) without the source/consumer wrapper
|
We want to have some real data in the quick-start instead of random values for demo purposes. Is there a replacement for the retired stream? If not, we should probably consider changing it to use the github events stream |
I don't think depending on a third party stream to gate PRs is a good idea, because they tend to change or can even be removed. |
Agree. We can probably add a quick-start to use this random event generator for testing purpose. |
| @@ -38,20 +36,17 @@ public MeetupRsvpJsonStream(boolean partitionByKey) | |||
| } | |||
|
|
|||
| @Override | |||
| protected MessageHandler.Whole<String> getMessageHandler() { | |||
| protected Consumer<RSVP> createConsumer() { | |||
There was a problem hiding this comment.
For the json stream, we actually expect the event to have nested json inside. With the current approach, all the fields are flattened
There was a problem hiding this comment.
Unfortunately, there's no record of what that JSON looked like. I'm trying to stop the CI build from failing so people can merge PRs.
The feed http://stream.meetup.com/2/rsvps was retired. As an external dependency, this should not be used to fail builds.